1. Field of the Invention
The present invention relates generally to image/video display and processing systems, and in particular, to a method, apparatus, and article of manufacture for conducting a hybrid vector/raster based paint operation that provides the ability to perform undo operations.
2. Description of the Related Art
In audio/video applications, a paint application enables the user to perform a wide range of painting tasks from simple retouching, to motion tracking, to full cartoon animation. For example, a user may paint (e.g., a brush stroke) onto a single particular frame. Prior art applications provide two different types of paint operations—vector and raster. In vector based paint operations, strokes are recorded in a vector for later use and manipulation. In raster based paint operations, the stroke is recorded as a bitmap or series of pixels that comprise the paint stroke. Vector-based paint operations are limited in that the method of drawing the stroke is not natural—a user must select a tool/type of paint stroke and utilize the tool in an unnatural manner. Further, as the number of vector-based paint operations/strokes increases, the ability to interact with the resulting image decreases. Raster based paint operations are limited in that the use of bitmaps consume considerable memory and processing such that unlimited undo operations are not feasible. In addition, prior art based paint operations cannot be performed on a clip containing multiple frames but are limited to a single static frame. These problems may be better understood with a description of prior art paint operations.
Within a paint application, an object consists of anything that may be drawn with a paint drawing operator. Objects may be divided into three main groups: paint, selection, and mask. Paint objects create or modify pixels. Selection objects define a specific area of the screen within which other paint objects or effects operate. Mask objects make areas of a layer transparent. Thus, after, an object is created, the object may be selected and its properties (e.g., size, shape, color, opacity, etc.) may be changed. Examples of objects that may be created in a paint application include: brush strokes, rectangles, ellipses, and polygons; text; lasso selections; magic wand selections; rectangular, elliptical, and polygon selections; text selections, fill; freehand masks; and rectangular, elliptical, and polygon masks. However, all of the above paint objects, whether vector or raster based have various limitations.
A vector based paint object is an image or stroke that is described by basic geometric shapes such as rectangles, polygons, circles, ellipses, lines, and curves. Accordingly, rather than recording all of the points/pixels that make up a rectangle or polygon, the vector image may be described using equations such as: the origin and radius for a circle; the origin, or the four vertices of a rectangle or square; the endpoints of a line, etc. Since a minimum amount of information is stored for a vector operation, undo operations may be easily supported. However, to draw a vector image/stroke, the user must first select the appropriate tool such as a circle tool, rectangle tool, Bezier curve tool, etc. Thereafter, the user would use the tool to specify the various points of the vector image. Such a use is not a natural way to draw or conduct a paint operation. In addition, because of the loss of “natural” painting capabilities, the quality of the resulting image may decrease.
A raster based paint object is an image wherein each pixel of the image is recorded/defined as part of the image. Additionally, when painting a raster-based stroke, the pixels of the original image may be modified based on the stroke thereby erasing/overwriting any prior raster image. Thus, a raster paint object changes the values of a buffer and the source. Further, to provide support for an undo operation, the entire raster image must be captured and stored in memory (e.g., random access memory) before/after each stroke is performed. In modern image processing systems, images are extremely large. Accordingly, recording multiple raster based images to provide the ability to perform an undo operation requires a considerable amount of memory and processing capabilities. Thus, the size and processing power needed to support an undo in a raster based paint operation may be impractical.
A paint application may be used in conjunction with a composite to paint on one layer or over several layers at a time. For instance, a paint application may be used to change a subject's eye color, remove wires, fix video drop-out on a selected layer, or create text effects and cartoon animations that are added over all of the layers in a composite. In addition, prior art paint applications include both vector paint applications (that define strokes as objects) and raster paint applications (that modify pixels in the image directly or write pixels directly in separate layers that may be combined in a source image thereby modifying the source image).
Paint operators are like any other operator in that they may be viewed schematically on a per-layer basis. FIG. 1 illustrates a process tree in a schematic view. A schematic view 100 uses nodes 102-114 to represent all of the operators in a workspace performed on a particular layer, and lines (edges) 116 between nodes 102-114 to indicate the flow of image data between the operators. The output of one node 102-114 is the source for the next node 102-114. As composites 112 or other branches (e.g., paint or color correction projects) are created, a process tree indicating the flow of data through the nodes 102-114 is built automatically. As illustrated in FIG. 1, source data consists of gator node 102 and snake node 104. A paint operation (e.g., via paint node 106) is performed on gator node 102 (e.g., by drawing the sun). The connection nodes 108-110 are then used as input to the composite operator node 112, that performs a compositing operation. The compositing operation node 112 produces a result/output 114.
In addition, prior art paint modules in procedural renderers (i.e., conducted in a schematic view) are raster paint nodes that do not have layers and modify their input buffers directly. In this regard, an input buffer in a procedural renderer may come from a fairly complex process tree and modifications to that tree will re-create the input buffer. A side effect of these modifications is that the paint strokes, being applied directly to the input buffer, are lost when that input gets modified through a change in the nodes upstream of a paint node.
As used herein, an operator (e.g., paint operator 106) is the most common node. A connection node 108 and 110 represents an input for an operator. In the case of a composite operator 112, the visible connection nodes 108 and 110 are layers. The output node 114 is a node representing the render queue settings for an operator. Lastly, several nodes may be grouped into a group node.
The process tree (such as the process tree illustrated in FIG. 1) can be used to manage operators and nodes. For example, users may add operators (e.g., a paint operation) to a branch, change the process order of operators 106, and navigate easily between operator controls. Further, if an operator is added to a layer in a schematic view 100, the operator will appear as a node (e.g., the paint node 106). The same operator node may also appear in other workspace environments (e.g., a workspace panel). Further, in the schematic view, particular operators may be turned on/off as desired simply by clicking on the icon for that operator (e.g., paint 106 or composition 112).
While the above prior art schematic view and paint operations may be used to modify and edit an image/clip, the prior art has many limitations. For example, in some prior art applications, a paint operator 106 is associated with a particular version of a source image 102. Thus, if a source image (e.g., image 102) is edited, the paint operator 106 disappears and all information associated with the paint node 106 is lost. In addition, prior art paint operations cannot be performed from within the schematic view 100. Thus, in order to perform any modifications using a paint operation, the prior art requires the user to exit the schematic view, enter the paint application, perform the desired paint operation, produce a result clip, reenter the schematic view 100, import the result clip into the schematic view 100 and continue.
In addition to the above limitations, modern audio/video processing applications such as editing and compositing applications perform operations on clips consisting of multiple frames. A user may create projects for painting on changing video frames and thereby apply a paint operation across multiple frames in a clip. Accordingly, the paint object must move and follow the action in a scene. Raster based processing of multi-frame paint operations are expensive in terms of processor and memory use. Further, vector based paint operations are not flexible or natural for a user. Accordingly, what is needed is a multi-frame paint based operation that is flexible in terms of use (similar to a raster based paint operation) yet allows an unlimited amount of undo operations without consuming excessive memory and processing.